Papers by Nagesh Panyam Chandrasekarasastry
Toward More Accurate and Generalizable Evaluation Metrics for Task-Oriented Dialogs (2023.acl-industry)
Copied to clipboard
Abishek Komma, Nagesh Panyam Chandrasekarasastry, Timothy Leffel, Anuj Goyal, Angeliki Metallinou, Spyros Matsoukas, Aram Galstyan
| Challenge: | Existing methods to dialog quality estimation focus on evaluating individual turns or collecting dialog-level quality measurements from end users immediately following an interaction. |
| Approach: | They propose a dialog-level annotation workflow called Dialog Quality Annotation . they propose to annotate dialogs for attributes such as goal completion and user sentiment . |
| Outcome: | The proposed model outperforms existing methods for dialog quality estimation . it shows that high-quality human-annotated data is important for dialog-quality evaluation . |